Evaluation of Artificial Neural Network Inference Speed and Energy Consumption on Embedded Systems

Digitalization and automation have been the driving force of the fourth industrial revolution and with them comes a vast amount of data that is collected and needs to be processed. Because of the high costs involved with operating data centers, the task of processing this data has been moved onto edge devices, which process the data on the spot and can act independently. In this paper, an accelerator for artificial neural network inference on edge devices, namely the Intel Neural Compute Stick (NCS), will be evaluated in terms of speed, energy consumption, and performance/cost ratio after which it will be compared to other similar solutions. It has been concluded that the inference speed is significantly better by using the Intel NCS than without it. This solution can be used in private projects and prototyping because of its low cost and low power consumption.